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ABSTRACT 


Given  a  directed  network  whose  arcs  have  lengths  unrestricted  In 
sign  and  which  contains  at  least  one  cycle ,  an  algorithm  to  find 
the  minimum  average  length  cycle  (length  divided  by  its  number 
of  arcs)  is  described.  A  direct  application  of  this  algorithm 
solves  the  problem  of  finding  whether  a  directed  graph  contains 
a  cycle  with  negative  length. 


CYCLES  WITH  MINIMUM  AVERAGE  LENGTH 


by 

Alain  Fllli&res 

1.  Notations  and  Definitions 

We  denote  a  directed  graph  (N,  r)  by: 

A  finite  set  N  of  nodes  1,  i  c  (1,  2,  A  mapping  T  : 

N  -*•  2'”  where  2^  is  the  set  of  all  subsets  of  N  .  J  e  r(l)  means  that 
there  exists  an  arc  (i,j)  going  from  node  i  to  node  j  .  We  will  denote  by 
y(i)  an  element  of  r(i)  ;  it  will  be  useful  to  write 

rk+1(i)  -  r(rk(i))  ,  k-o  ,  1,  2,  ... 

Let  A  -  {(i,j)/ieN  ,  jer(i)}  be  the  set  of  arcs.  A  mapping  l  : 

A  -*■  1R  is  given,  Hi,j)  is  called  the  length  of  arc  (i,j)  .  We  define 

*(K)  -  l  Ha) 
aeK 


where  K  is  any  subset  of  A  . 

Note  that  lengths  are  unrestricted  in  sign. 

We  will  assume  in  the  following  that: 

(N,  r)  contains  at  least  one  cycle. 

(N,  r)  is  so  that  T(i)  4  ♦  VieN  .  This  is  no  real  restriction  since  one 
can  always  add  to  the  set  A  an  arc  (i,i)  with  a  length  Hi,i)  ■  ®  for  every 
1  which  has  r(i)  -  $  ,  in  the  original  graph. 

A  p-arce  cycle  is  a  set: 

C  -  { 1 ,  y(i),  Y^(i),  ...»  yP(D  ■  i)  where  yP(D  is  an  element  of  I*p(i)  . 


( 


The  average  length  of  the  p-arcs  cycle  C  Is  defined  by 


Let  C  -  (1,3, 4,1) 


7-5+12  14 

3  "  3  * 


This  paper  describe,  an  algorithm  which  yields  the  minimm*  average  length 

oyole  in  the  graph  (N,  f)  defined  above.  Thus  this  algorithm  can  also  be  uaed 

to  find  out  whether  the  graph  contains  a  negative  length  cycle.  The  method 
is  an  application  to  a  deterministic  case  of  a  policy-iteration  method  for  multiple 

Markov  chain  processes  with  rewards  [2],  [3]  . 

2.  Policy  y 

A  policy  y  is  a  mapping  y  :  N  +  N  that  asaociatee  to  every  UN  an 
element  y(i)cr(i)  • 

*  To  get  the  maximum  average  length  cycle,  we  replace  the  original  arc  lengths  l 
by  -2,  and  apply  the  same  algorithm. 


Let  (N,y)  be  the  subgraph  of  (N,y)  representing  a  policy  y  .  The  set 
of  arcs  of  (N,y)  is: 

Ay  -  { (i, j)/ieN,  j  -  y(i)}  . 

There  are  n  |  r(l)  |  different  policies  with  an  equal  number  of  associated  graphs 
ieN 

(N,y)  ,  where  |  r(i)  |  is  the  cardinality  of  the  set  r(i)  . 

3.  Properties  of  Graphs  (N.v) 

F.l  (N,y)  breaks  down  into  connected  components  »  hi:{l,  2,  ...»  k} 

such  that: 

H4  n  N  -  ♦  i,je{l,2 . k) 

k 

„UA  *  ■ 

h-1 

P.2  (N^,  y^)  ,  he{l,  k}  contains  one  and  only  one  cycle. 

P.3  Every  ieNh  ,  hc{l,  . . . ,  k}  is  either  a  node  of  the  unique  cycle  of 

(Nh,  y^)  ,  or  a  node  of  a  unique  chain  leading  to  this  cycle. 

These  properties  are  an  obvious  consequence  of  the  definition  of  y  and  the 
fact  that  N  is  a  finite  set. 

k  is  the  number  of  connected  components  of  (N,y)  . 

There  is  a  one-to-one  correspondence  between  the  set  of  cycles  defined  by 
Y  »  and  the  connected  components  of  (N,y)  . 

v 

Let  us  call  C!  the  cycle  in  the  connected  component  containing  node  i  . 


5 


Y 

when  gi  is  the  average  length  of  the  cycle  . 

y  ts  better  than  y'  »  Y^  y'  »  if  g  <  g'  . 

Y  is  optimal  if  y  -C  Y  for  *11  Y  • 

Y  and  y '  are  equivalent  if  and  only  if  y(i)  ■  Y*(i)  •  VicN  . 

A 

In  other  words,  cj  is  the  DiniDiim  average  length  cycle  which  can  be  reached 

* 

from  i  in  the  initial  graph  (N,T)  .  Therefore  (N,y)  contains  the 
minimum  average  length  cycle  of  (N,T)  . 

The  principle  of  the  policy  -  Iteration  procedure  is  to  start  with  an 
arbitrary  policy  and  to  improve  it  step  by  step  (according  to  the  criterion  JL 
described  above)  until  optimality  is  reached. 

5.  Functional  Relation 

A  policy  y  being  given,  we  have  a  first  relation 


U)  8l  -  SY(1)  VicN 

where  y(1)  Is  associated  with  i  by  the  policy  y  . 

Let  X1(t)  be  the  length  of  a  t-arcs  chain  starting  from  i  in  (N,y)  ; 
X1(t)  satisfies: 

(2)  A^t+1)  -  l(i,Y<i)>+AY(1)(t)  VieN  . 


It  is  important  to  note  that  we  are  not  dealing  with  the  problem  of  finding  the 
minimum  average  length  cycle  which  goes  through  a  given  node.  An  algorithm  for 
this  case  would  solve  the  Traveling  Salesman  Problem  by  adding  an  arbitrary  large 
number  to  the  length  of  every  arc  leaving  this  node. 


If 


t  is  sufficiently  large,  from  properties 


one  can  see  that 


st*rting  from  any  node  i 
general : 


and  coming  along  a  large  number  of  arcs  one  goes  in 


a)  along  a  unique  chain  up  to  a  node,  say  m  , 

b)  a  large  number  of  times  around  the  cycle  starting  from  m  , 

c)  stop  at  a  node  of  the  cycle,  say  m* 


So  Xi(t)  can  be  expressed  in  the  following  form: 

Ai  (c>  *  ^(p)  +  [■~£]*(c)+x  , 

i  r  J 

when 


p  >  0 

r  • 


A 


i.  th.  renter  of  ere.  between  1  end  m  ,  which  corresponds  to 
the  number  of  arcs  of  the  cycle 

the  distance  between  m  and  m'  along  cj 


(a) 


For  a  large  value  T  of  t 


[^]*(C)'f  1(C) 

or  T  times  the  average  length  of  cj  .  Besides  the  quantity:  X^(p)  + 
that  US  will  call  depend  only  on  i  for  a  fixed  T  . 

Hence,  we  get  the  relation: 


(3)  X4  (T)  -  Tgt  +  for  T  large  . 

By  using  (3)  In  (2),  we  get: 

(T+l)gt  +  wA  -  1(1, y(1))  +  Tg^(i)  +  wy(1) 


or  using  (1) 


gt  +  wA  -  1(1, y(D)  +  w^(i)  . 

Then,  we  get  the  following  relations  satisfied  by  any  policy  y  : 

(4)  tt  -  8y(1) 

(5)  gj  +  »i  -  1(1, y(i))  + 

foT.-  all  ieN  . 


(4)  and  (5)  give  n  equations  in  the  n+k  unknown  variables  g, ,  g 

1  n 


and 


w^ ,  . . . ,  .  However,  we  only  need  to  know  the  value  of  the  differences 


w  -  w  for  i»  j  in  the  same  connected  component.  So  we  set  w  -  0  for 
1  J  *h 

an  arbitrary  i^  in  each  connected  component,  h  »  1,2,  ....  k  ,  and  we  call  v 

the  relative  value  of  w4  obtained  by  this  way. 

The  value  g^  and  v^  associated  with  every  node  ieN  for  a  given  policy, 
are  used  to  determine  a  better  policy  than  y  . 


6.  Alaorithm 

Initial  policy  yQ 

YQ  can  be  arbitrarily  choosen.  A  good  starting  policy  is  yq  satisfying: 

l(i,y  (D)  -  Min  [i(i,h)]  VieN 

her(i) 

Let  Yk  be  the  policy  choosen  at  step  k  -  0,1,2,  ...  and  g^(i) ,  vk<i)  ,VleN 
the  values  defined  in  Section  5  corresponding  to  Yk  • 

Iteration 

The  step  k  +  1  which  yields  a  better  polciy  Y^+i  proceeds  in  two  phases. 
(I)  Solve  the  following  system  for  gk(i)  and  vk(i)  : 

(6) 

(7) 

Note  that  in  practice  the  set  of  equations  (6)  is  not  necessary,  a  policy 
Yk  being  given  it  is  easy  to  compute  directly  the  value  gk(i)  for 


gk(i)  “  gk(Yk(1)) 


i  ™  1 ,  « •  •  |  n 


gk(i)  +  vk(i)  -  l(i,Yk(i))  +  vk(Yk(D)  . 


i  ■  1,  ...,  n  and  replace  them  in  (7) 


(II)  Yk+^  is  obtained  by  letting  Yk+^(i)  ,  i  -  1,  ...»  n  ,  in  the  following 
way: 

Let  h  be  satisfying: 

(8)  g.  (h)  -  Min  (g.(j))  . 

Jer(i)  11 

Case  1: 

If  h  is  unique  choose: 

W1*  ’ h  • 

Case  2: 

If  h  is  not  unique,  choose  an  arbitrary  Yk+1(i)  satisfying: 

(9)  Ui,Yk+1(i))  *  vk*Yk+l(i^  "  +  vk+l(J^  * 

In  both  Case  1  and  Case  2,  use  the  following  rule  (R) : 

(R)  If  Yk(i)  satisfies  (8)  and  (9),  set  Yk+1(i)  ■  Yk(i) 

The  rule  (R)  means  that  if  the  node  associa  3d  to  i  in  the  step  satisfies 
(8)  and  (9)  that  will  be  the  node  associated  to  i  in  the  k+l1*1  step. 

Note  that  if  (N,yk)  has  only  one  connected  component,  test  (8)  can  be 
skipped. 

The  tests  (8)  and  (9)  applied  to  every  node  i,  i  -  1,2,  ...,  n  ,  give  a  new 
policy  Yk+i  •  At  this  stage,  there  are  two  possibilities: 
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(a) 


Yk+1(i)  -  >K(i)  VieN,  then  Yk  -  Y 


(b) 


3i  such  that  Yk+^(i)  +  Yk(i)  *  t*len  80 
to  (I)  with  k  ■  k  +  1  . 


Remarks 

1)  The  test  (1)  implies  that  the  comparisons  of  the  quantities  £ ( i , j )  +  vk(j) 
are  made  with  j  belonging  to  the  same  connected  component,  hence,  the  sense  of 
the  relative  values:  v^  of  the  w^  . 

A  A 

2)  If  (N,D  is  strongly  connected,  y  is  such  that  (N,y)  contains  only 
one  connected  component  and  hence  has  a  unique  minimum  average  length  cycle. 

3)  At  each  step  of  the  procedure,  in  particular  at  the  last  one,  one  knows 
the  immediate  descendent  of  each  node,  hence,  it  is  easy  to  get  the  cycles. 

7.  Proof  of  the  Algorithm 

We  need  to  prove  the  two  following  statements: 

1)  If  for  any  i  belonging  to  any  cycle  of  (N»Yk+1)  Yk(D  i  Yk+1(i) 
then  8k+1(i)  <  8k(D  •  (From  the  properties  of  (N»\+l^  that  implies 

8k+l  -  8k(1)  VieN)  * 

2)  If  Yk(i)  ■  Yk+1^  VieN  then  Yk  ■  Y  •  That  means  that  when  the 
procedure  stops,  one  cannot  find  a  policy  which  leads  to  a  better  value  of  the 

minimum  g(i)  ,  VicN  .  (Rule  (R)  implies  the  procedure  stops  on  a  finite  number 
of  steps.) 

Proof  1: 

Let's  assume  y,  (i  )  t  y.,,(i  )  for  at  least  one  i  belonging  to  a  cycle 

iC  O  iC » X  0  0 
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of  (N,Yk+1)  •  The  tests  (1)  and  (2)  of  phase  II  Implies: 

<10>  8k(rk+l(1))  -  «k  ■  »i 

where  <  0 

(11)  *<i,Yk+1(i))  +  vk+1  (Yk+1(i))  -  i(i,Yk(D)  -  vk(Yk(D)  -  ^ 

where  <  0  . 

Note  that  from  the  Rule  (R)  : 

W°  ‘  V‘>  • 0 

Tk+i(1)  *  1rk<1)^*i " 0 
Phase  I  leads  to  the  systems: 

(12)  \  gk(i)  -  gk(Yk(D) 

(13)  (  gk(i)  +  vk(D  -  l(i,Yk(D)  +  vk(Yk(i)> 

(14)  \  8k+l(i)  "  8k+l(Yk+l(i)) 

(!5)  I  gk+1(D  +vk+j(i)  "  fc(i,Yk+1(i))  +  vk+l(Yk+l(i)) 


VieN 


VieN 


By  introducing:  ^  ,  Agk(i)  -  gk+1(D  "  gR(i)  ,  ^vR(i)  -  vk+1(i)  -  vk(i)  , 


12 


in  the  differences  (14)  -  (12)  and  (15)  -  (14),  we  get: 


(16) 


6gk(i)  -  ♦1  +  A8k(Yk+1(D) 


(17) 


Agfcd)  +  Avkd)  ■  ♦1  +  Avk^Yit+i^1^ 


for  all  ieN  ;  ^  ^  <  0 


Let  be  the  r-cyd«  of  the  hth  connected  component  of  (N»Yk+1> 

contains  iQ  satisfying  the  hypothesis,  i.e.,  <p  <0  . 

o 

By  adding  (16)  for  iet^  : 


lc 


I  4^(1)  -  I  +  I  4gk(Yt+1(i» 


l£Ch  lECh 


Then,  ][  <|».  -  0  which  implies  ip  -  0  , 

leCh 


Hence: 


(18) 


4gk(i)  -  A8k(Yk+1(i» 


By  adding  (17)  for  ieC^  and  using  (18),  we  get 


which 


VleCh  ’ 


Vi£Ch  • 


rA8k(D  m  l  *± 

l£Ch 


♦t  1  0  • 


From  the  hypothesis,  there  exists  1  eC^  such  that  <p^  <0  the  R.  H.  S.  of 

o 

(10)  Is  negative  which  implies: 


Agk(i)  <  0 


VicGh 


This  proof  applies  for  all  h  satisfying  the  hypothesis,  Q.  E.  D  . 

Proof  2: 

Let  y  ■  yfl  be  the  policy  obtained  at  the  end  of  the  procedure.  Let  us 
suppose  there  exists  a  policy  Yt  such  that: 

gt(i0>  <  8a(*0)  for  least  one  iQ 
since,  did  not  come  out  from  the  procedure,  we  have: 

(10')  88(Yt(i))  -  88<Yt(i)  •  nt  n4  >  0 

(11')  vg(Yt(i))  +  * (i,Yt(i))-vg  (Ys(i))-f  (i»Yg(i))  ■  >  0 

for  all  ieN  . 


Besides,  we  get  the  equivalent  relation  as  in  Proof  1,  with  k  -  s  ,  k  +  1  «  t  . 
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Ats8(1)  "  8t(i)  ‘  g8(i) 

At8V(1)  "  Vt(1)  ‘  V8(i) 

<16'>  Atfl8(i^  "  ni  +  At88(Yt(i)^ 

(17')  4tag(i)  +  Atgv(i)  “  Wj  +  Atgv(Yt(l)) 

ni  -  0  ’  wi  -  0  * 

L«t  C'h  be  the  p'-  cycle  of  (N,Yt)  which  contains  i  ;  by  adding 
(16')  and  (17')  for  ieC'h  ,  we  get  as  in  Proof  1: 


then, 


since 


p'Ats8(i) 


Atsg(i)  >  0 


Wi  >  0 


Hence,  the  contradiction. 


8.  Example: 


The  value  of  Ykd) ,  gfc(i) ,  »k<i)  ,  k-0,  1,  2,  3,  4,  1-1,2 . 10 

given  in  the  following  tableau  (computations  have  been  made  by  hand)  . 


are 


Y 


The  minimum  average  length  cycle  is  (1,4, 5 ,3,1)  ;  its  average  length  is 
This  cycle  is  unique,  y ^  ■  Yj  has  only  one  connected  component. 

Remarks : 

1)  shows  that  the  original  graph  has  at  least  a  negative  length  cycle 

2)  We  set  arbitrarily  vq(6)  -  vq(10)  -  0  components  of  yq  .  and 
vk(10)  -  0  ,  k  -  1,2, 3, 4  . 

3)  y2  »  Y3  did  not  Improve  the  value  of  g  because  the  nodes  of  the 
cycle  (3, 9, 7, 3)  kept  the  same  immediate  descendents  during  Steps 


2  and  3  . 
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